Add instructions to run coverage #737

j-rivero · 2020-06-02T23:07:52Z

The PR adds instructions to run coverage builds. It uses the subset of ros2.repos file (by now at my own fork I'll update the URL as soon as I send the PR for it to ros2/ros2).

Status is draft pending on @maryaB-osr's and @chapulina review and final URL location.

The goal is to provide users with a method to call ci_linux_coverage job in a way that the Jenkins build can exercise all testing mechanisms present in all the ROS2 core packages. The set in this commit is still a work in progress and will be modified.

…bout how to read the results

chapulina

Thanks, Jose!

source/Contributing/ROS-2-On-boarding-Guide.rst

Co-authored-by: Louise Poubel <louise@openrobotics.org>

j-rivero · 2020-06-04T14:18:01Z

In the case we want to add instructions to run the coverage calculator script, they are in the branch coverage_set_ros2_script

j-rivero · 2020-06-04T14:47:56Z

made some improvements on formatting, ready to go once we have the final URL for coverage.repos file.

dirk-thomas · 2020-06-04T15:16:31Z

Instead of creating and maintaining another .repos file the set should be generated using rosinstall_generator.

j-rivero · 2020-06-04T15:45:04Z

Instead of creating and maintaining another .repos file the set should be generated using rosinstall_generator.

That sounds reasonable to me. I use it with my set of proposed packages and found some issues:

The following unreleased packages/stacks will be ignored: test_cli, test_cli_remapping, test_communication, test_launch_ros, test_quality_of_service, test_rclcpp, test_security, test_tf2

Seems to work with released packages, how should I include the repositories for test_* packages?

I assume that we need to store the set of proposed packages for coverage in a place and generate it dynamically somehow? at running time in Jenkins? on changes on ros2.ros2 repository?

dirk-thomas · 2020-06-04T16:20:54Z

Seems to work with released packages, how should I include the repositories for test_* packages?

If a repository is not part of the distribution.yaml file then rosinstall_generator can't fetch it. For system_tests it might be best to just add a source entry for it. Then I would assume rosinstall_generator should be able to fetch it with --repos or --upstream-development.

Co-authored-by: Marya Belanger <marya@openrobotics.org>

j-rivero · 2020-06-05T00:13:51Z

Please double check some of the edits I made to make sure I didn't change any intended meaning

You did not change any meaning but fix a good bunch of errors, thanks @maryaB-osr

chapulina

The text looks good, I'm just debugging the get_coverage_ros2_pkg.py script directly with @j-rivero

source/Contributing/ROS-2-On-boarding-Guide.rst

Co-authored-by: Louise Poubel <louise@openrobotics.org>

maryaB-osr

Missed a couple little things, but otherwise lgtm!

source/Contributing/ROS-2-On-boarding-Guide.rst

Co-authored-by: Marya Belanger <marya@openrobotics.org>

source/Contributing/ROS-2-On-boarding-Guide.rst

dirk-thomas · 2020-06-05T14:37:17Z

source/Contributing/ROS-2-On-boarding-Guide.rst

+
+ * From the ci_linux_coverage Jenkins build copy the URL of the build
+ * Download the `get_coverage_ros2_pkg <https://github.com/j-rivero/ros2_coverage_jenkins_params/blob/coverage_checker/get_coverage_results/get_coverage_ros2_pkg.py>`__ script
+ * Execute the script: ``./get_coverage_ros2_pkg.py <jenkins_build_url> <ros2_package_name>`` (`README <https://github.com/j-rivero/ros2_coverage_jenkins_params/blob/coverage_checker/get_coverage_results/README.md>`__)


Should we really recommend "external" scripts like this which have never been reviewed and are not accessible to edit by anyone else in the team?

@j-rivero is working on moving it to a more common location. Do you have a preference?

I reviewed the script while reviewing this PR.

is working on moving it to a more common location. Do you have a preference?

I don't have a preference - just not on a personal GitHub user. Please suggest proposals.

I reviewed the script while reviewing this PR.

Just a view things I notices while briefly looking at it (and since I don't see any PR on that repo I will just post it here):

shebang line doesn't use /usr/bin/env

incorrect usage handling

error messages don't go to stderr

7 line convoluted logic to determine a simple boolean

parentheses around conditional

I don't have a preference - just not on a personal GitHub user. Please suggest proposals.

Fully agree on moving it a better location. ros2/ci repository is the first one that comes to my mind.

Just a view things I notices while briefly looking at it (and since I don't see any PR on that repo I will just post it here):

Thanks for looking into it Dirk. I've updated the code with your findings but let's start a PR procedure to look into it deeper.

j-rivero added 3 commits May 15, 2020 20:09

Add instructions to run coverage step by step and include a section a…

9537ad0

…bout how to read the results

Update instructions to use a .repos file

08878a0

j-rivero marked this pull request as draft June 2, 2020 23:07

j-rivero requested a review from maryaB-osr June 2, 2020 23:08

mjcarroll temporarily deployed to ros2-documentation-pr-737 June 2, 2020 23:08 Inactive

j-rivero requested a review from chapulina June 2, 2020 23:08

chapulina requested a review from nuclearsandwich June 3, 2020 00:36

chapulina reviewed Jun 3, 2020

View reviewed changes

j-rivero and others added 4 commits June 3, 2020 14:02

Update source/Contributing/ROS-2-On-boarding-Guide.rst

0555302

Co-authored-by: Louise Poubel <louise@openrobotics.org>

Update source/Contributing/ROS-2-On-boarding-Guide.rst

79b0fbe

Co-authored-by: Louise Poubel <louise@openrobotics.org>

Update source/Contributing/ROS-2-On-boarding-Guide.rst

88678fe

Co-authored-by: Louise Poubel <louise@openrobotics.org>

Update source/Contributing/ROS-2-On-boarding-Guide.rst

0dc590e

Co-authored-by: Louise Poubel <louise@openrobotics.org>

mjcarroll temporarily deployed to ros2-documentation-pr-737 June 3, 2020 12:03 Inactive

Add instructions to calculate the coverage rate

9450ee9

mjcarroll temporarily deployed to ros2-documentation-pr-737 June 3, 2020 14:14 Inactive

j-rivero added 2 commits June 4, 2020 16:15

Add instruction to use coverage rate checker

a438fb9

Missing final of sentence

6f7fb29

Improve formatting

9a5c089

mjcarroll temporarily deployed to ros2-documentation-pr-737 June 4, 2020 14:29 Inactive

j-rivero added 3 commits June 4, 2020 16:29

Merge branch 'coverage_set_ros2' into coverage_set_ros2_script

329a376

Improve formatting

161f6a0

Add internal anchors to coverage sections

186bf88

mjcarroll temporarily deployed to ros2-documentation-pr-737 June 4, 2020 14:43 Inactive

j-rivero mentioned this pull request Jun 4, 2020

Add source entry for system_tests in foxy ros/rosdistro#25332

Merged

Update source/Contributing/ROS-2-On-boarding-Guide.rst

1e2cbea

Co-authored-by: Marya Belanger <marya@openrobotics.org>

mjcarroll temporarily deployed to ros2-documentation-pr-737 June 5, 2020 00:08 Inactive

Fix sentence, thanks to Marya

3325c7a

mjcarroll temporarily deployed to ros2-documentation-pr-737 June 5, 2020 00:12 Inactive

Update source/Contributing/ROS-2-On-boarding-Guide.rst

691bc90

Co-authored-by: Marya Belanger <marya@openrobotics.org>

mjcarroll temporarily deployed to ros2-documentation-pr-737 June 5, 2020 00:12 Inactive

j-rivero and others added 3 commits June 5, 2020 02:12

Update source/Contributing/ROS-2-On-boarding-Guide.rst

c9494a6

Co-authored-by: Marya Belanger <marya@openrobotics.org>

Update source/Contributing/ROS-2-On-boarding-Guide.rst

eeda9b4

Co-authored-by: Marya Belanger <marya@openrobotics.org>

Update source/Contributing/ROS-2-On-boarding-Guide.rst

cf9cbad

Co-authored-by: Marya Belanger <marya@openrobotics.org>

mjcarroll temporarily deployed to ros2-documentation-pr-737 June 5, 2020 00:13 Inactive

j-rivero requested review from dirk-thomas, maryaB-osr and chapulina and removed request for nuclearsandwich June 5, 2020 00:17

chapulina approved these changes Jun 5, 2020

View reviewed changes

source/Contributing/ROS-2-On-boarding-Guide.rst Outdated Show resolved Hide resolved

Update source/Contributing/ROS-2-On-boarding-Guide.rst

3dc83fb

Co-authored-by: Louise Poubel <louise@openrobotics.org>

mjcarroll temporarily deployed to ros2-documentation-pr-737 June 5, 2020 00:34 Inactive

maryaB-osr approved these changes Jun 5, 2020

View reviewed changes

j-rivero and others added 4 commits June 5, 2020 02:40

Update source/Contributing/ROS-2-On-boarding-Guide.rst

368ef92

Co-authored-by: Marya Belanger <marya@openrobotics.org>

Update source/Contributing/ROS-2-On-boarding-Guide.rst

194fc7c

Co-authored-by: Marya Belanger <marya@openrobotics.org>

Update source/Contributing/ROS-2-On-boarding-Guide.rst

bd12928

Co-authored-by: Marya Belanger <marya@openrobotics.org>

Update source/Contributing/ROS-2-On-boarding-Guide.rst

15d6aee

Co-authored-by: Marya Belanger <marya@openrobotics.org>

mjcarroll deployed to ros2-documentation-pr-737 June 5, 2020 00:41 View deployment

chapulina merged commit 2d74b95 into master Jun 5, 2020

delete-merged-branch bot deleted the coverage_set_ros2 branch June 5, 2020 00:59

dirk-thomas reviewed Jun 5, 2020

View reviewed changes

source/Contributing/ROS-2-On-boarding-Guide.rst Show resolved Hide resolved

source/Contributing/ROS-2-On-boarding-Guide.rst Show resolved Hide resolved

source/Contributing/ROS-2-On-boarding-Guide.rst Show resolved Hide resolved

j-rivero mentioned this pull request Jun 5, 2020

Style corrections to coverage notes #751

Merged

dirk-thomas reviewed Jun 5, 2020

View reviewed changes

j-rivero mentioned this pull request Jun 16, 2020

Add script to calculate coverage rates ros2/ci#476

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add instructions to run coverage #737

Add instructions to run coverage #737

j-rivero commented Jun 2, 2020

chapulina left a comment

j-rivero commented Jun 4, 2020

j-rivero commented Jun 4, 2020

dirk-thomas commented Jun 4, 2020

j-rivero commented Jun 4, 2020

dirk-thomas commented Jun 4, 2020

j-rivero commented Jun 5, 2020

chapulina left a comment

maryaB-osr left a comment

dirk-thomas Jun 5, 2020

chapulina Jun 5, 2020

dirk-thomas Jun 5, 2020 •

edited

Loading

j-rivero Jun 8, 2020

Add instructions to run coverage #737

Add instructions to run coverage #737

Conversation

j-rivero commented Jun 2, 2020

chapulina left a comment

Choose a reason for hiding this comment

j-rivero commented Jun 4, 2020

j-rivero commented Jun 4, 2020

dirk-thomas commented Jun 4, 2020

j-rivero commented Jun 4, 2020

dirk-thomas commented Jun 4, 2020

j-rivero commented Jun 5, 2020

chapulina left a comment

Choose a reason for hiding this comment

maryaB-osr left a comment

Choose a reason for hiding this comment

dirk-thomas Jun 5, 2020

Choose a reason for hiding this comment

chapulina Jun 5, 2020

Choose a reason for hiding this comment

dirk-thomas Jun 5, 2020 • edited Loading

Choose a reason for hiding this comment

j-rivero Jun 8, 2020

Choose a reason for hiding this comment

dirk-thomas Jun 5, 2020 •

edited

Loading